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Abstract 

Problem Statement One way to delineate the main characteristics of 
effective teaching within the higher education system is to gather college 
students' opinions of an effective instructor. Research based on students' 
perceptions of efficient teaching revealed a series of teaching behaviors 
setting the distinction between good and poor teaching. However, studies 
also indicate differences across culture, and in Turkey, there has been little 
research on the topic. 

Purpose of Study : The goal of this study was to determine instructor and 
course characteristics and teaching dimensions that discriminate between 
instructors who received the highest and the lowest student ratings within 
a Turkish college setting, by incorporating both quantitative and 
qualitative methods. 

Method: A total of 23,814 students across different departments in the 
university rated 630 instructors on a scale developed to assess students' 
perceptions of instructors' performance. In addition, students were asked 
to respond to an open-ended question to provide their own impressions of 
each instructor. Then, students' ratings were analyzed by means of 
discriminative functional analysis, and written statements provided by 
students were analyzed via content-analysis techniques by using a 
combination of manual and computer-assisted methods (NVivo 9). 

Findings and Results: According to the quantitative analyses, although 
course and instructor characteristics were weak in discriminating the 
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groups, all teaching dimensions (relationships with students, effective 
teaching, exams and evaluation, contribution to generic skills, class 
interaction, and organization and planning) were very useful in 
discriminating the instructors who received the best and the poorest 
ratings. Also, qualitative analysis revealed 4 themes consistently 
distinguishing the two groups: lecturing, relationship with the students, 
knowledge and expertise, and exams and evaluation. 

Conclusions and Recommendations : This study replicates the existing 
literature on student perceptions of effective teaching, with a culturally 
different, large sample. It also adds support to the notion that there are 
teaching behaviors, such as lecturing skills, fair evaluations, respect and 
interest toward students, and demonstrating expertise, that help draw the 
distinction between good and poor teaching in the eyes of students and 
that could therefore assist the improvements efforts of teacher education. 

Keywords: Instructor effectiveness, poor teaching, college teaching, student 
evaluations 


Introduction 

One of the important preconditions to ensure higher education quality is to 
employ effective instructors and maintain instructors' excellence through students' 
educational years. Additionally, teacher quality has been linked to long-term 
achievement going beyond school years (Chetty, Friedman, & Rockoff, 2014). Hence, 
attempts to define and delineate the main characteristics of the best instructors 
within a higher education system accumulated numerous studies. The most basic 
form of this research is conducted through asking college students to rate the most 
important features of an "effective or ideal" instructor on a researcher-provided 
scale. In such a study with a semantic differential task, Pozo-Munoz, Rebolloso- 
Pacheco, and Femandez-Ramirez (2000) asked students to rate an ideal teacher based 
on a 7-point rating scale with two bi-polar adjectives listed on opposite ends (e.g., 
nice-unpleasant; expert-inexpert, etc.). They found four distinguishable factors 
defining ideal teachers: teaching competency, teachers' qualities, appearance, and 
directiveness. Zhang, Fike, and Dejesus (2015) reported that the highest rated 
qualities of ideal instructors were being knowledgeable and grading fairly. 

More recent studies combining both survey and qualitative methods asked 
students both to rate the likelihood of each statement to define an ideal teacher on a 
scale and to list the most important features of their ideal teacher. In such a study, 
Okpala and Ellis (2005) reported five main components of quality teaching at higher 
education: caring for students and their learning, teaching skills, content knowledge, 
dedication to teaching and love of work, and good verbal skills. Khandelwal (2009) 
reported six dimensions: encouragement, course preparation and delivery, fairness, 
rapport with students, spending time with students outside of class, and control. 
Slate, LaPrairie, Schulte, and Onwuegbuzie (2011) identified four reoccurring themes: 
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knowledgeability, understanding, communication skills, and teaching well. 
However, they also detected some differences between undergraduate and graduate 
students' definitions in that graduate students placed more importance on being 
knowledgeable, connecting with the real world, passion for the job, and flexibility. In 
a comprehensive review, Feldman (2007) synthesized the current research on teacher 
efficiency and reported that the factors most related to teaching effectiveness varied 
depending on whether the students' own achievement or the results of the overall 
teaching evaluation is taken as the indication of teacher effectiveness. Within Turkish 
college settings, Tunca, Alkin-Sahin, Oguz, and Bahar-Guner (2015) obtained five 
themes by analyzing students' definitions of ideal instructors, namely the role and 
responsibilities of instructors, values, personal characteristics, social responsibilities, 
and ethical principles. These findings indicate that although some teacher behaviors 
are more salient and are better descriptors of effective teachers, still students' choices 
of best teachers do not necessarily correspond to best learning outcomes, indicating 
that students' characteristics and expectations might also affect the definition of 
excellent teachers. 

Indeed, studies in which participants come from culturally different orientations 
other than Western countries seem to further support this possibility. In an earlier 
study. Bail and Mina (1981) compared American and Filipino students' ratings of 39 
statements pertaining to instructor behavior according to their perceived importance 
for general teaching effectiveness. Although both groups perceived three dimensions 
as equally important (instructors' competence on the subject matter and their ability 
to relate it to class materials, quality of feedback and evaluation procedures, and 
instructor-student rapport), Filipino students placed more importance on qualities 
pertaining to authority and personal appearance for effective teaching than did 
American students. Watkins and Akande (1992) reported that although there were 
similar patterns of findings with Western studies, Nigerian students might hold a 
more "general view" of the instructors rather than distinguishing different aspects of 
teaching. Miller, Dzindolet, Weinstein, Xie, and Stones (2001) reported that U.S. 
students endorsed higher importance to teacher preparation, evaluation, and 
presentation items and opportunities for student inquiry in defining effective 
teaching than South African students, who in turn endorsed higher importance to 
these items than Chinese students. Taken together, these studies indicate that 
although there are identifiable themes consistent across studies, there are also 
differences of student opinion as to the best instructors, based on culture. Still, in 
spite of countless studies on the topic in the world's literature, only a few of them 
have been conducted in the Turkish culture. Therefore, it remains a need to define 
best-teacher qualities in the eyes of students. The topic is especially important within 
Turkish culture since it nests a variety of interesting contradictions compared to 
Western culture, where the majority of knowledge about teaching qualifications is 
obtained. Foremost of these contradictions comes from the recent years' radical 
change of educational policies. 

The Turkish educational system has only recently discovered constructivist 
teaching (Akpinar & Ay din, 2007) and, despite great challenges — such as teacher 
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readiness needing to be reconciled with parental expectations for higher student 
scores on standardized tests such as the TEOG, YGS, and KPSS —efforts to establish 
an educational system based on constructivism remain strong. Constructivism 
requires students' active participation, good cooperative skills, as well as motivation 
for and enjoyment from learning activities. However, in a changing world of 
expectations and rules, students' surroundings that shape their motivation, skills, 
and even personalities have also been changed. Students are growing up in a 
technological world where there is less opportunity to practice social skills and fewer 
responsibilities at home, and exams push students to adopt a more results- 
orientation rather than focusing on learning and enjoyment. Within such an 
educational system hosting many contradictions and challenges, it is not easy for 
teachers to easily understand the students' expectations or define the teacher 
practices that best fit to these expectations. Hence, in spite of an already accumulated 
knowledge base on teaching quality, both the existence of differences across studies 
and the unique complexities of Turkish educational settings warrant further inquiry 
in this area. 

Further, although much research has accumulated, a majority of it focuses on 
identifying the qualities of teachers who are defined as the best. Only a few compare 
the features of instructors delineated as either the best or the poorest within the same 
context. These studies revealed that organization of content, providing variety, 
knowledge, creating an enjoyable learning experience, and communication skills 
were the most commonly stated attributes of the most effective instructors, while 
poor course organization, poor communication, unfairness, and being boring were 
the most defining characteristics of the worst instructors across studies (e.g.. Check, 
2001; Epting, Zinn, Buskist, & Buskist, 2004; Fortson & Brown, 1998; Johnson, 1990; 
Young & Shaw, 1999). However, the majority of these studies —with the exception of 
only a few—asked students to define their best/worst instructors rather than to 
actually to rate their best or worst instructors (e.g.. Marsh, 1977; Young & Shaw, 
1999). Students might apply different criteria for rating a real instructor as opposed 
to defining an imagined one; when students are asked to rate their best instructors, 
they inadvertently activate their implicit schemas for an ideal instructor, causing 
some bias on ratings. For example, students might implicitly hold an ideal instructor 
schema that is more organized, highly knowledgeable, and highly dedicated to 
teaching, but they might award a higher rating to an easy grader, or a fun-loving or 
understanding instructor. Therefore, in this study students were asked to rate each of 
their instructors' performance and to provide an open-ended explanation of what 
they thought about the instructor without judging whether he or she is a good or 
poor instructor. Then, those instructors who received the highest evaluation points 
for three consecutive semesters were identified as the highest rated instructors. 
Likewise, those who received the lowest scores for three consecutive semesters were 
identified as the lowest rated instructors. The two instructor groups were then 
compared based on demographics such as age, years of experience, gender, academic 
rank, class size, instructors' total number of students, and total workload (as the 
number of total hours spent teaching courses in a week) to analyze if demographic 
variables would change the ratings. Although past studies have already indicated 
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that neither students' nor instructors' demographics have much effect on student 
evaluations (Marsh, 1984), in Turkey instructors have much higher workloads and 
student density than in Western universities where the literature mostly derives 
from. More importantly, in Turkey, unlike in Western countries, there is no 
determining utility of students' evaluations or even good teaching skills on hiring 
and promoting decisions. Hence, it is possible that those who devote more time to 
academic behaviors, such as publishing, rather than teaching behaviors, such as 
preparing for the course or caring for the students, might attain a higher academic 
rank compared to those who spend more time on teaching activities that leaves less 
room for academic duties. To restate, in spite of the existing studies to distinguish 
poor and effective teaching in college, the changing face of the educational needs of 
students as a result of technological advances, cultural needs, and the 
methodological differences across studies necessitates further research to define 
teaching-efficient criteria in the eyes of Turkish college students. Therefore, this 
study aims to extend current knowledge on teaching quality by comparing the most 
salient features of the instructors who received the highest/lowest evaluation points 
across the university within a Turkish higher educational setting, by using both 
qualitative and quantitative methods. 

In summary, the main purposes of this study were to determine: (a) Which 
demographics (if any) discriminate those instructors who received the highest 
evaluations and those who received the lowest evaluations; (b) which instructional 
dimension(s) best discriminate those instructors who received the highest 
evaluations and those who received the lowest evaluations; and (c) whether there 
are identifiable patterns on students' written statements that distinguish the 
instructors who received the highest evaluations from the instructors who received 
the lowest evaluations, and do they support the quantitative analyses? 


Method 


Research Design 

The students were asked to evaluate the performances of all instructors who 
lectured them in the Fall semester of the 2009-2010 school year, on a scale published 
on the school's website two weeks before final exams started. Students were also 
asked to respond to an open-ended question, which required students to provide 
their own impressions of the corresponding instructor in order to contribute to the 
improvement efforts of the university's instructional quality. The data were also 
collected at the end of the next two semesters. That is, feedback was collected in the 
2009-Fall, 2010 Spring, and 2011 Fall semesters. An instructor was classified under 
the highest rated category if his or her evaluations were 1 standard deviation above 
the mean for three consecutive semesters. Likewise, an instructor was classified 
under the poorly rated category only if her or his evaluations were 1 standard 
deviation lower than the mean for three consecutive semesters. Using these criteria, 
27 instructors were classified as the highest rated and 32 instructors were classified as 
the lowest rated. There were 26 male and 6 female instructors in the lowest rated 



122 


Sevgi Ozgiingor & Erdinc Duru 


group, and there were 17 male and 10 female instructors in the highest rated group. 
For the lowest rated group, 6.2% were professors, 21.9% associate professors, 50% 
assistant professors, and 21.9% lecturers; for the highest rated group, 22% were 
associate professors, 51.9% assistant professors, and 25.9% lecturers. 

Research Sample 

This study used the data collected as a part of the "Pamukkale University 
Teaching Staff's Instructional Process Evaluation and Improvement Project." The 
project started in the 2009-2010 Fall semester, and data were obtained from all 
students attending to the university with the exception of medical school students. A 
total of 23,814 students (12,142 men, 51%, and 11,672 women, 49%) evaluated 630 
instructors. There were 412 male and 218 female instructors. The ages of the 
instructors ranged from 23 to 64, with a mean of 40. 30 (ss=7. 97). Years of experience 
of the instructors ranged between less than 1 year to 33 with a mean of 10. 33 (ss=4. 
76). 

Research Instrument 

Demographics. Demographics including students' GPA, instructor gender, 
course load, instructors' total number of students and class size, academic rank, age, 
years of experience, and discipline were obtained from the university's data 
processing unit. 

Evaluation of instructors' teaching and educational processes. Students' 
evaluations of the instructors' performances were measured by a scale developed 
through modification of the most widely used questionnaires in the student 
evaluations literature, such as Course Experience Questionnaires (Ramsden, 1991) 
and SEEQ (Marsh, 1980, 1984; Marsh & Dunkin, 1997), based on the university's 
educational system's features and needs. Although the new scale was most closely 
related to SEEQ, its modifications involved reducing the number of items on all 
subscales and the addition of a subscale to measure students' perception of 
instructors' role on the acquisition of higher-order thinking skills such as critical 
thinking, new viewpoints, and flexibility in thinking. The resulting scale had 20 items 
compromising 6 subscales. Students were also asked to respond to two additional 
items: One structured item asking students to evaluate the instructor-made difficulty 
levels of the course after considering the innate nature of the course itself, and one 
open-ended question to write any useful information about the instructor's 
performance or how to improve it. The first subscale (Effective Teaching) taps the 
instructor's teaching skills such as being able to capture student interest and making 
the content meaningful and valuable; the second subcomponent (Course 
Organization and Planning) reflects the instructor's ability to organize and deliver 
the course content in a fluent and comprehensive manner; the third subscale (Exams 
and Evaluation) was made up from the items measuring the fairness and 
appropriateness of an instructor's evaluation practices; the fourth component 
(Relationship With Students) included items measuring the nature and closeness of 
instructors' communication skills and relationships with students; the fifth subscale 
(Class Interaction) taps the instructors' energy and behaviors to encourage students' 
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involvement in class activities; and, finally, the last subscale (Contribution of Generic 
Skills) measured instructors' ability to support development of students' thinking 
skills, such as critical thinking and gaining new viewpoints. 

Validity and Reliability 

Both test-retest and internal reliability values calculated to establish the scale's 
reliability levels were satisfactory, ranging between .70 and .98 for all subscales. 
Factor analysis initially revealed one dimension indicating a general view of 
instruction rather than capturing a multidimensional nature of the teaching 
behaviors. However, when principal component analysis was forced to create 6 
dimensions, it revealed 6 subcomponents explaining 86% of total variance with all 
the items loading only on the expected dimensions. 

Procedures and Statistical Analysis 

Results were organized around the study's main purposes and presented 
accordingly. First, discriminate functional analysis was performed on the data to 
determine which (if any) demographics, as well as six teaching dimensions, 
differentiate between the instructors who were awarded the highest scores and those 
receiving the lowest evaluations. Second, a qualitative content analysis was run by 
using NVivo 9 to determine the possible repeating patterns in describing the highest 
and the lowest rated instructors. Finally, both qualitative and quantitative results 
were combined conceptually to synthesize the findings. 


Findings and Results 

Discriminant Analyses 

At the first step of the statistical analysis, a discriminate functional analysis 
indicated that only three of the demographics—which were years of experience, 
student's course grade, and class size—were significant across all demographic 
variables; however, significant differences existed for all teaching dimensions 
between the two instructor groups. 

The discriminant function produced a large canonical correlation of .97 and a 
coefficient of determinism of 100. The largest correlation with the discriminant 
function was Contribution of Generic Skills, followed by Class Interactions, Course 
Organization and Planning, Effective Teaching, Relationships with the Students, and 
Exams and Evaluations, in order. All three demographics had a negative 
contribution to discriminant function. The discriminant function for the stepwise 
analysis resulted in a correct classification of 100% of the respondents into their 
respective groups with no misclassification. The results are given in Table 1. 
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1 

Table 1. 



Discriminant Function Analysis of Demographics and Teaching Dimensions between 
Instructors Who Received the Best Evaluations and Those Who Received the Poorest 
Evaluations (N=59) 

Part A: Correlation with Discriminant Function Variable 



Study Variables 

Correlations 


Contribution to generic skills 

.863 


Class interaction 

.799 

Instructional 

dimensions 

Course organization and 

planning 

.773 


Effective teaching 

.766 


Relationships with students 

.757 


Exams and evaluation 

.702 


Student's grade 

-.083 


Class size 

-.073 


Instructor's gender 

.051 

Demographics 

Academic discipline 

.038 


Difficulty level of the course 

-.033 


Instructor's total course hours 

-.022 


Instructor's total students 

number 

-.020 


Academic rank 

.019 


Instructor's age 

.005 


Years of experience 

-.110 
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Part B: Tests of Equality of Group Means 


Independent Variables 

Wilks 1 

Lambda 

F 

dfl 

df2 

P 

value 

Effective teaching 

.093 

553.827 

1 

57 

.000 

Class interaction 

.086 

603.545 

1 

57 

.000 

Relationships with 

students 

.095 

541.009 

1 

57 

.000 

Contribution to 

generic skills 

.075 

704.281 

1 

57 

.000 

Exams and evaluation 

.109 

464.993 

1 

57 

.000 

Course organization 

and planning 

.092 

565.220 

1 

57 

.000 

Student's grade 

.897 

6.524 

1 

57 

.013 

Class size 

.919 

5.042 

1 

57 

.029 

Difficulty level of the 

course 

.982 

1.023 

1 

57 

.316 

Instructor's gender 

.958 

2.499 

1 

57 

.119 

Instructor's age 

1.000 

.028 

1 

57 

.868 

Academic discipline 

.977 

1.353 

1 

57 

.250 

Instructor's total 

course hours 

.992 

.438 

1 

57 

.511 

Years of experience 

.832 

11.527 

1 

57 

.001 

Instructor's total 

students number 

.993 

.386 

1 

57 

.537 

Academic rank 

.994 

.341 

1 

57 

.562 
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Part C: Canonical Discriminant Function 


Eigenvalue 

Canonical 

Correlation 

Chi-square 

df 

P value 

16,575 

97.7 

140.458 

16 

.000 


Part D: Classification Result 


Actual Group Predicted Group Membership 

Membership Best Evaluated Poorest Evaluated 

Best Evaluated 27 0 

Poorest Evaluated 0 32 


Percent of cases correctly classified = 100% 


Qualitative Analysis of the Data 

The written statements were analyzed via the content-analysis technique by using 
a combination of manual and computer-assisted methods (NVivo software for 
qualitative data management). Within this process, words and sentences that are 
conceptually similar were organized, coded, and interpreted to create meaningful 
themes and subthemes. After the organization of the codes, obtained themes and sub 
dimensions were expressed in percentages and given in a frequency table. The 
students' written statements were analyzed separately for the two instructor groups. 
There were a total of 945 responses for the open-ended question for the instructors 
who received the highest evaluations, and there were 1,393 responses for the 
instructors who received the lowest scores. However, only a total of 1,895 responses 
were codeable since most of them consisted of general statements such as "the 
lecturer was awesome," "this class sucks," or "worst teacher ever" without further 
explanations as to why. Below is a summary of the themes related to each group and 
the categories related to these themes (Table 2 and Table 3). 

For the instructors who were rewarded the highest points, all the codeable 
statements were categorized into five themes: Lecturing, relationship with the 
students, knowledge and expertise, personality, and exams and evaluation. Lecturing 
characteristics further compromised six subthemes: teaching skills and techniques, 
value and utility of the content, enjoyment and interestingness of the lessons, 
instructor's positive attitude toward teaching and students, preparing students for 
the profession, and development of critical thinking and new viewpoints. 
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Table 2. 


Number and Percent of the Frequencies of the Themes for the Highest Rated Instructors 


Themes 

Categories 

Number 

of students 
mentioned 
the theme 

% of the 

students 

mentioned the 
theme 

Lecturing 

Teaching skills and 
techniques 

160 

28.9 

Value and utility 

96 

17.35 

Enjoyment and 

Interestingness 

71 

12.83 

Attitude toward teaching 
and the students 

35 

6.33 

Preparing students for 
their profession 

32 

5.78 

Critical thinking and 
new viewpoints 

15 

2.71 


TOTAL 

409 

73.96 

Relationships with 
the students 


56 

10.12 

Knowledge and 

expertise 


40 

7.23 

Personality 


25 

4.52 

Exams and 

evaluation 


23 

4.15 


TOTAL 

553 

100 


For the instructors who received the lowest evaluations, a total of five dimensions 
were obtained: lecturing, examinations and evaluations, content, relationships with 
the students, and expertise and knowledge. Lecturing had six subcomponents: 
inefficient or inadequate teaching skills, rote teaching, boring teaching style, 
providing insufficient examples, negative attitude toward the lessons, and inability 
to teach according to students' developmental levels, in the order of importance. 
Exams and evaluations had four subthemes: too difficult or above students' 
developmental levels, not fair, exams based on only factual information rather than 
deep understanding, and wanting more than what was given. Content compromised 
three subthemes: too hard, unnecessary or useless, and outdated. Students' responses 
about the relationships with the students theme revolved around two main 
subthemes: frequent use of insults and criticism, and negative attitudes towards 
students including the use of threats to make the exams or course content too 
difficult in order to maintain authority within the class. 
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Table 3. 


Number and Percent of the Frequencies of the Themes for the Poorly Rated Instructors 


Themes 

Categories 

Number of 

students 
mentioned the 
theme 

% of the students 
mentioned the 

theme 

Lecturing 

Skills (lack of) 

264 

19.67 

Rote teaching 

148 

11.03 

Boring 

136 

10.13 

Providing insufficient 
example 

62 

4.62 

Negative attitude toward 
teaching 

44 

3.28 

Inability to teach 
according to students' 
developmental level 

20 

1.49 

TOTAL 

674 

50.22 

Exams and 
evaluations 

Difficulty 

levels/appropriateness to 
students' levels 

97 

7.23 

General comments 
/fairness 

75 

5.59 

Exam based on factual 
information 

48 

3.58 

Wants more than what is 
given 

40 

2.98 

TOTAL 

220 

19.37 

Content 

Inappropriate difficulty 
level 

140 

10.43 

Lack of utility 

63 

4.69 

Content is outdated 

17 

1.27 

TOTAL 

260 

16.39 

Relationships 
with the 

students 

Insults and criticism 

79 

5.89 

Negative attitudes toward 
students 

90 

6.71 

TOTAL 

169 

12.59 

Expertise and 
knowledge 


19 

1.42 


GENERAL TOTAL 

1342 

100 
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When themes obtained from students' responses for instructors who received the 
highest scores and lowest scores were compared, it appears that obtained themes and 
subthemes were very closely matched for the groups. For both groups, the most 
frequently mentioned theme was related to the instructors' lecturing characteristics 
including teaching skills and techniques, which made the course effective or 
ineffective. More than 70% of the statements for the highest rated instructors and 
more than 50% of the statements for the lowest rated instructors were about the 
lecturing characteristics. While the statements for the instructors who received the 
highest evaluations emphasized the efficiency, joyfulness, usefulness (in a variety of 
ways, such as increasing critical thinking skills, knowledge, adaptation to real life), 
and gaining vocational skills, as well as a respectful and valuing attitude toward 
both the course and the students, statements for the lowest rated instructors were 
about a lack of teaching skills causing memorization rather than learning, the 
inability to consider students' developmental levels during instruction, boring 
lecturing, and a negative attitude toward both students and to the content being 
taught. Likewise, although the ordering based on mentioned frequency was different 
for each group, statements regarding both groups involved student and instructor 
relations, knowledge and expertise levels of the instructors, and exams and 
evaluations. With the exception of the exams and evaluation theme, for all themes, 
statements for each group were in essence bipolar statements. That is, the highest 
rated instructors were cherished for being caring, kind, and respectful to students 
and to their ideas, while the lowest rated instructors were described as being rude, 
inconsiderate to students and their needs, and disrespectful to their ideas. However, 
for the exam and evaluations, both groups' statements echoed the toughness of the 
content; while the highest rated instructors' students perceived this toughness as a 
necessary component of understanding and learning, students of the lowest rated 
instructors thought the toughness was created artificially by inefficient teaching 
behaviors, making it an obstacle to learning rather than a help. 


Discussion and Conclusion 

The purposes of this study were to determine which instructor and course 
characteristics and teaching dimensions best differentiate the instructors who 
received the highest evaluation points from those instructors who received the 
lowest evaluation points for three consecutive semesters, incorporating both 
quantitative and qualitative methods. For this aim, students' evaluations collected 
through a teaching evaluation scale were analyzed and compared to the obtained 
themes derived from students' open-ended statements. Quantitative analyses 
indicated that although the discriminative powers were very little, three 
demographics were useful in discriminating two groups, namely instructors' 
experience in years, the student's grade in the course, and the number of students in 
the course. Previous studies usually found little or no effect of either instructor or 
students demographics (e.g.. Marsh, 1984; Beran & Violato, 2005), with some 
exceptions. This study's findings are compatible with previous findings since, 
although there were some effects, these effects were very little for all demographics 
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including the strongest of all three, which is experience. These results are in line with 
earlier studies which found a small but inverted U-shaped quadratic relation in 
which instructors receive higher evaluations initially, level up at some early point, 
and then decline slowly thereafter (Marsh & Roche, 2000). However, some cultural 
differences in academic settings might also be accountable for this result since the 
establishment of higher academic standards in the higher education system became 
important in Turkey only recently. Therefore, younger instructors might be more 
knowledgeable and inclined to use active learning techniques, and be more 
understanding toward students both because it is an important element of active 
learning and because they share a similar conceptualization of authority due to lesser 
cohort differences. In terms of class size, past research usually shows a significant but 
small effect of class size on student ratings (Hanushek, 2002). This study further 
extends the current findings by showing that, although small, this effect is also 
applicable for distinguishing between good and poor instructors in the eyes of 
students; the number of students in the class might cause an otherwise a regular 
instructor to be viewed as a very poor instructor. Likewise, the inefficiency of a poor 
instructor can be no longer compensated by the students' own efforts within 
crowded classes. 

As to students' grades, findings on this issue are more complicated and less 
agreed upon. While some studies found significant correlations between students' 
evaluations and course grades (Cashin, 1995), some others found no or little 
relationship between these two (Marsh & Roche, 1999). Further, researchers' 
interpretation of the meaning of the existing link differ in that some consider this link 
as a sign of validity of the ratings since better instructors' students should learn 
better, resulting higher grades (Marsh & Roche, 1997). Others consider this link to be 
a sign of threat to the validity of student ratings, however, since high evaluations are 
attributed to students' appreciation of grading leniency (Greenwald & Gillmore, 
1997). This study's findings add further support to the validity of students' ratings 
since, within the efficient teaching literature, the best instructors are usually defined 
as challenging, someone who sets higher standards and pushes students to do their 
best (e.g.. Acker, 2003; Slate et al., 2011). Both qualitative and quantitative analyses of 
this study were in line with this notion. According to qualitative analysis, students in 
both groups frequently mentioned that the course was difficult and that the 
instructor had high expectations. However, the explanations for the difficulty levels 
were entirely different. Students thought that the instructors who garnered the 
highest ratings provided a rich learning experience and expected the best from the 
students in return. In contrast, students believed that the poorly rated instructors 
made the course harder by asking useless questions, relying on rote teaching, and 
under-teaching or avoiding certain concepts. According to the students, the highest 
rated instructors not only delivered the content very efficiently, but also they held 
high expectations for better performance. Students seem to perceive the high 
expectation for success as fair and useful as long as instructors themselves create the 
necessary conditions for it. On the other hand, poorly rated instructors' exams seem 
to be perceived as difficult only because they are unfair in the sense that these 
instructors ask more than what they provide and their exams contain rote learning 
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and memorization with useless content, making it hard to learn and get motivated to 
learn—which was described as unfair by the students. 

While only a few demographics were useful in discriminating the two groups, all 
teaching dimensions were important in the classification of the groups. Teaching 
dimensions in order of the highest contributions to the discriminate variance were 
generic skills, class interaction, course organization and planning, effective teaching, 
relationships with the students, and exams and evaluations. These results are 
important in the sense that they lend further support to previous studies of teaching 
efficiency with data obtained from students with, culturally, a very different 
orientation. Good instructors might possess a variety of characteristics, however, 
some of these characteristics keep reappearing across studies: good teaching skills 
(using a variety of teaching methods, giving examples to illustrate concepts, 
increasing student interest and value), good organization and preparation, 
instruction in an enjoyable and enthusiastic way, being knowledgeable, caring 
deeply for students and teaching, having a passion and commitment to teach, being 
fair in exams and grading, creating a comfortable learning environment, and 
emphasizing and using active learning (e.g., Fortson & Brown, 1998; Khandelwal, 
2009; Slate et ah, 2011). Likewise, although much more limited, literature on poor 
instructors demonstrates that poor instructors also possess some characteristics 
which are comparable across studies. Poor instructors are boring (Fortson & Brown, 
1998; Check, 2001), unable to deliver the content because of a lack of organization 
(Fortson & Brown, 1998; Check, 2001) or a lack of knowledge or ability (Check, 2001), 
and unable to relate to students due to poor communications skills (Johnson, 1990; 
Epting et al., 2004). In addition to being disorganized, they are unfair in evaluation 
practices (Johnson, 1990; Khandelwal, 2009). The current study's findings, combined 
with the existing literature, indicate that efficient and poor instructors are not very 
different from each other. Rather, they are the flip side of the same coin—meaning, 
that although there is no certain definition of good teaching, there are some elements 
that are very useful when possessed and very harmful when missing for efficient 
teaching, at least in the eyes of students. 

Another contribution of this study was through converging quantitative and 
qualitative results to provide a better understanding for the distinction between the 
lowest and highest rated instructors. According to this analysis, although qualitative 
results were mainly supportive of the quantitative analyses, there were dimensions 
obtained from qualitative analyses not captured by the sole use of the scale, such as 
instructors' personality, the encouragement of the development of critical thinking, 
and knowledge and expertise levels. Patrick (2011) reported that the dimensions of 
openness and conscientiousness were the best personality predictors of both course 
and instructor ratings. The most frequently mentioned features of the instructors in 
written statements of the current study were also closely related to 
conscientiousness, openness, and agreeableness. Based on these findings, it is 
possible that there might be a relationship between an instructor's personal resources 
and the academic, developmental, and psychological needs of students. In other 
words, while the conscientiousness dimension of the highest rated instructors' 
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personalities might serve to meet the academic needs of students, the openness and 
agreeableness dimensions of these instructors' personalities might serve to meet the 
interpersonal needs of students, including their developmental and psychological 
requests. Taken together, these findings seem to lend further support to Chickering 
and Reisser's (1983) theory, which postulates that the postsecondary experience has 
the potential to foster feelings of intrinsic motivation by fulfilling students' 
autonomy, competence, and relatedness needs. 

In short, this study replicates the existing literature on teaching effectiveness with 
a culturally different and large sample, and adds clear support that there are actual 
teaching behaviors that really draw a distinction between good and poor teaching, 
and therefore could assist efforts for quality improvements in a higher education 
system. 
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En Yiiksek ve En Diisiik Ogrenci Degerlendirmelerini Alan Ogretim 
Elemenalarim Ayiran Demografik ve Ogretim Elemani Ozellikleri 


Atif: 

Ozgungor, S. & Duru, E. (2015). Course and Instructor Characteristics Distinguishing 
Highest and Lowest Student Ratings of Instructors. Eurasian Journal of 
Educational Research, 61,118-136. http://dx.doi.org/10.14689/ejer.2015.61.7 


Ozet 

Problem Durumu: Yiiksekogretimde kaliteyi saglamanm en onemli yolu ogrencileri 
gelecekteki mesleklerine hazirlayan egitim kadrosunun kaliteli ve yetkin olmasmi 
saglamaktir. Bu ytizden ytiksekogretimde etkili ogretim elemanmi tammlamaya 
yonelik pek gok ara§tirma yapilmi§tir. Bu ara§tirmalarda kullanilan en temel yontem 
iiniversite ogrencilerinden ideal, etkili ya da en iyi ogretim elemanlarmi 

tanimlamalarmi istemektir. Bu gali§malarda ytiksekogretimde etkili ogretim 
elemanmi tammlayan ozelliklerin ba§mda ogrenci ve ogrenme diizeylerini 

onemseme, ogretme becerileri, alan bilgisi, ogretmeyi sevme ve i§ini sevme, dersin 
iyi planlanmasi, adil degerlendirme gibi ozellikler siralami§tir. Ancak var olan 
gali§malar, sonuglarm lisans ve yiiksek lisans ogrencileri igin farklila§tigim ve 
ogrencilerin etkin ogretim elemanlarma ili§kin dti§uncelerinin farkli ktilttirel yapilara 
gore degi§tigini gostermi§tir. Alan yazmda ogrencilerin tanimladigi etkili ogretim 
elemani ozelliklerinin ktilttire gore degi§tigini one siiren gali§malara ragmen 
yiiksekogretimde etkili ogretim elemani tammlamasina ili§kin bulgularm neredeyse 
tamanu yurtdi§mda yapilan gali§malara dayalidir. Oysa son yillarda teknolojik, 
kiiltiirel ve egitim politikalari agismdan gati§an ozellikleri barmdiran Tiirkiye igin bu 
konunun ara§tirilmasi daha da onemlidir. §oyle ki, bir taraftan egitim 

politikalarmdaki degi§imler ogrencilerin aktif katilimini gerektiren yapilandirmaci 
egitimi te§vik ederken, bir taraftan TEOG, LGS, KPSS gibi smavlar ogrencileri daha 
hazirci ve ezberci olmaya itmektedir. Bu ttir geli§kileri aym anda barmdiran egitim 
sisteminde gorev yapan egitimciler igin ogrencilerin goztinde en yararli ve bu 
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ytizden ogrenmeye giidtileyici ogretmen davram§larmin ne oldugunun belirlenmesi 
daha onemli hale gelmektedir. Ayru zamanda, Iilkemizde ogretim elemanlarmm 
tizerindeki ogrenci sayisi ve ders ytikiiniin daha yogun olmasi gibi unsurlar goz 
oniinde bulunduruldugunda ogretim elemanlarirun performanslarmda ki§isel 
ozellikleri di§mdaki ozelliklerin de belirleyici oldugu du§iiniilebilir. 

Ara$tirmanin Amaci: Bu gali§manm temel amaglari (a) iiniversite ogrencileri 
tarafmdan sunulan performans degerlendirmelerinde en ytiksek ve en dti§tik 
puanlari alan ogretim elemanlarmi ayri§trran demografik ozelliklerin belirlenmesi (b) 
iiniversite ogrencileri tarafmdan yapilan performans degerlendirmelerinde en 
ytiksek ve en dii§iik puanlari alan ogretim elemanlarmi ayiran ogretim etkinliklerine 
ili§kin boyutlarm belirlenmesi (c) ogrencilerin en yiiksek ve en dti§uk puanlari 
verdikleri ogretim elemanlarma ili§kin agik uglu soruya yonelik 
degerlendirmelerinin analiz edilerek iki grubu ayrrmada i§levsel olan anlamli 
oriinttiler olu§turulup olu§turulamayacagmin belirlenmesidir. 

Ara$tirmanin Yontemi: £ali§ma kapsammda veriler Tip Fakiiltesi harig Pamukkale 
Universitesi'ne bagli turn akademik birimlere devam etmekte olan toplam 23814 
(12142 erkek ve 11672 kiz) ogrenciden derslerine giren ogretim elemanlarinm 
performanslarmi tiniversitenin web sitesinde yaymlanan bir olgek iizerinden 
degerlendirmeleri istenerek elde edilmi§tir. Kullamlan olgek Marsh tarafmdan 
geli§tirilen SEEQ'nin Tiirkge uyarlamasmi kapsasa da, gali§manm amacma uygun 
bazi degi§imler de igermektedir. Bu degi§imler madde sayismm kisaltilmasi ve 
ogrencilerin bili§sel becerilerinin geli§imini destekleyici davrani§lara yonelik yeni bir 
boyutun eklenmesini igermektedir. Nihai olgek alti alt boyut igermektedir: etkili 
ogretim, smif igi etkile§im, ogrencilerle ili§kiler, planlama, olgme ve degerlendirme 
ve genel becerilere katki boyutlari. Veriler 2009 Giiz, 2010 Bahar ve 2011 Gtiz 
donemlerinin sonunda final haftasmdan onceki 2 hafta siirecinde toplanmi§tir. 
Ogrencilere aym zamanda degerlendirdikleri turn ogretim elemani igin dii§iincelerini 
belirtmeleri istenen agik uglu bir soru sunulmu§tur. Ug donemin sonunda, her 
donem igin standart sapmanm 1 yukarismda ve 1 a§agismda degerlendirmeler alan 
ogretim elemanlari belirlenmi§tir. Bu §ekilde tig donemde standart sapmanm 
yukarismda ve a§agismda puan alan ogretim elemanlari belirlenerek ogrenciler 
tarafmdan performanslari en iyi ve en kotii olarak algilanan ogretim elemanlari 
tespit edilmi§tir. Bu smiflama sonucu toplam 630 ogretim elemamnm 32'si (26 erkek, 
6 kadm) en dii§iik degerlendirmeleri alan, 27 (17 erkek, 10 kadm) ogretim elemani 
da en yiiksek puanlari alanlar olarak siniflandinlmi§tir. Ogrencilerin agik uglu 
yanitlari NVivo 9 programi kullanilarak igerik analizi yontemiyle analiz edilmi§tir. 

Ara$tirmanin Bulgulan: Ara§tirmanm birinci amacma yonelik olarak en ytiksek 
puanlari alan ogretim elemanlari ile en dti§tik puanlari alan ogretim elemanlarmi, 
ogretim elemamnm deneyimi, cinsiyeti, toplam ders ytikti, ogrenci sayisi, unvani, 
ogrencinin ders ba§ari notu, akademik birimi, smif mevcudu ve dersin zorluk 
derecesini igeren demografik ozelliklerin ayirmada yararli olup olmadigmi 
belirlemek amaciyla discriminant analizi yapilmi§tir. Analiz, sadece 3 demografik 
degi§kenin (deneyim, ogrenci ba§an notu ve smif mevcudu) gruplari aynmada etkili 
oldugunu, ancak bu etkinin zayif oldugunu gostermi§tir. Ara§tirmamn ikinci 
amacma yonelik olarak yapilan analizler ogretim etkililigine ait ttim alt boyutlarm 
gruplari ayirmada etkili oldugunu ortaya koymu§tur. Son olarak, ogrencilerin agik 
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uglu yamtlanmn igerik analizi yoluyla incelenmesi sonucu en ytiksek puanlari alan 
ogretim elemanlarmi tammlayan 5 alt boyut elde edilmi§tir; dersin i§leni§i, 
ogrencilerle ili§kiler, uzmanlik, ki§ilik ve olgme degerlendirme. Benzer §ekilde, en 
dii§iik puanlari alan ogretim elemanlarma yonelik yanitlar dersin i§leni§i, olgme ve 
degerlendirme, igerik, ogrencilerle iligkiler ve uzmanlik olmak tizere yine 5 alt boyut 
olu§turmu§tur. 

Ara$tirmamn Sonuglan ve Onerileri: Bu gali§manm temel amaci takip eden 3 donem 
boyunca ogrenciler tarafmdan en yiiksek ve en dii§iik degerlendirmeleri alan 
ogretim elemanlarmi ayri§trran demografik ve ogretim etkinligi boyutlarmm 
belirlenmesidir. Yapilan analizler demografik degi§kenlerden iki grubu ayri§tirmada 
etkili ozelliklerin ogretim elemammn deneyimi, sinif mevcudu ve ogrenci ba§ari 
puam oldugunu ve ogretim etkinligi alt boyutlarmm tamammm iki grubu 
ayri§tirmada etkili oldugunu gostermi§tir. Ogrencilerin agik uglu cevaplarma yonelik 
yapilan igerik analizleri nicel bulgulari destekler niteliktedir. Analizler sonucu en 
yiiksek puan alan ogretim elemanlarmi tanimlayan 5 alt boyut olu§turmu§tur. 
Ogrenci algilarma gore, en yiiksek puanlari alan ogretim elemanlari, dersi i§lerken 
farkli ve eglenceli yontemler kullanan, ogrencileri aktif kilan, derste ogrencilerle 
siirekli etkile§im iginde olan, alanmda uzman, ogrencilere ve ogrenci geli§imine igten 
ilgi duyan, yaptigi i§ten haz alan ve bunu smif igi hareketleriyle siirekli sergileyen, 
ogrencilerin mesleki ve ki§isel geli§imlerini destekleyen ve diinyaya ve insanlara 
kar§i pozitif tutumlariyla ogrencilerin ya§ama baki§mda radikal degi§iklikler yaratan 
ogretim elemanlandir. Ayrica bu ogretim elemanlarmm smavlan zor ancak adil 
olarak tammlanmaktadir. En dti§iik puan alan ogretim elemanlari genelde aym 
ozelliklerin yokluguyla tammlanmi§tir. Ders i§leyi§i sirasmda sikici bir tarz 
sergileyen, sunu teknigi kullanan, kitabi okuyarak ya da ezberden tekrarlayarak, 
ogrenci katilimmi saglamayan bu ogretim elemanlari aym zamanda igerigi 
gtincelle§tirememeleri ya da ogrencilerin giinliik ve mesleki ya§amlariyla 
ili§kilendirilmemeleri nedeniyle ele§tiri almi§lardir. Alanmda yetersiz olarak 
algilanan bu bireyler, derse ve ogrencilere kar§i olumsuz, a§agilayici davram§lariyla 
tanimlanmi§tir. Son olarak, bu ogretim elemanlarmm anlatmadiklari konulari smava 
dahil ederek, yeterli ornek gozmeyerek, ezber sorarak smavlan zorla§tirdiklan ve 
adil olmak yerine ki§isel ilgi ve ihtiyaglarma gore puanlandirma yaptiklari 
belirtilmi§tir. Ozetle, bu gali§manm bulgulari, ogrencilerin en yiiksek ve en du§uk 
puan verdikleri ogretim elemanlarmm alan yazmda yer alan etkili ogretmen 
tanimiyla paralel oldugunu ve bulgularm yiiksekogretimde egitim kalitesini 
artirmada yararli olacagmi gostermektedir. 

Anahtar Sozcukler: Ogretmen etkinligi, yiiksekogretimde egitim kalitesi, ogrenci 
degerlendirmeleri, iyi ogretmen 



